This disclosure relates generally to a vision system, and more specifically to an image processing system that stabilizes 360-degree video data.
Consumer-level 360-degree cameras are becoming increasingly popular and are often used in a variety of user devices to capture 360-degree images and/or videos, and 360-degree video systems generate a completely immersive experience that transports a user to another world in which a user can look in any direction in a scene. In one example, multiple cameras are used to in a Virtual Reality (VR) system to create 360-degree virtual scenes for a user to experience a virtual scene.
But problems may arise when recording and generating 360-degree video data. As one example, casual 360-degree video data taken with a hand-held camera often look shaky, and individual frames are geometrically distorted by rolling shutter wobble. Shake in 360-degree video data is particularly severe because it causes discomfort when watched using VR headsets. Furthermore, most existing video stabilization algorithms that resolve problems of shaky 360-degree video data are designed specifically for narrow field of view video and use two-dimensional (2D) motion models; however, many current 2D motion models do not translate easily to the spherical warp-around domain in 360-degree video data. Additionally, although some current algorithms reconstruct three dimensional (3D) models of camera trajectory and scene geometry to realize stabilized 360-degree video data, performing a full 3D reconstruction is a complex process, slow in practice, and not robust under certain situations, such as the absence of translation motion.
Therefore, there is a need for a new algorithm that enables 360-degree video data stabilization with higher robustness, speed, and smoothing ability.